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ABSTRACT 

The Western Region AMIDS Evaluation Team examined a 
variety of approaches to leadership development which would make it 
possible for adult, business, technical and vocational educators to 
become knowledgeable in evaluation. The following three intents were 
specified for the leadership development project: (1) to provide 

participants with an introductory pregtam in evaluative theory and 
methodology, (2) to develop basic evaluative skills concerning 
techniques for evaluation ot on-going pcograras in adult, business, 
technical and vocational education, (3) to provide participants with 
sufficient reference materials and direction to enable them to 
undertake subsequent, individual study activities in the area ot 
evaluation. A one-day workshop was planned for 21 participants. An 
inductive-deductive learning approach was selected tor both the 
workshop and the individual study phase of this leadership 
development activity. During the first hour of the workshop, a "case 
model" was distributed. The afternoon activities started with a team 
teaching session dealing with the concepts of performance standards 
and standards ot excellence. The last hour of the workshop was 
devoted to a presentation ot evaluation techniques and methodology 
appropriate to adult, business, technical and vocational education. 
In general the participants were very positive in their reactions to 
the workshop. (CK) 
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FOREWORD 

The areas of Adult, Vocational and Technical Education have recently 
experienced a revival of interest and concern for evaluation and accounta- 
bility. In particular, and of major interest, is the development of new and 
unique ideas about evaluation which lend themselves to popular adaptation in 
the programs of adult, vocational and technical education. 

Accordingly the California State Department of Education, Vocational 
Education Section, in cooperation with the Division of Vocational Education, 
University of California, formali?.ed a depth study of Leadership Development 
in Evaluation for the purposes of improving general accountability in adult, 
vocational and technical education, and requested Professor James A. Farmer, 
Jr., Graduate School of Education, University of California, Los Angeles, to 
provide the leadership in developing further some of his evaluation principles 
and practices. 

Involved in the evaluation study are members of the EPDA Leadership 
Development Program who ere in residence as Graduate students at UCXA under 
the provisions of Sec. 552, Part F, Education Professions Development Act, 

P. E. 90-35. 



Melvin L. Barlow, Director 
Division of Vocational Education 
University of California 

Professor of Education 
University of California 
Los Angeles 



Richard S. Nelson, Chief 
Program Operations Unit 
Vocatio' Education Section 
Califor .a State Dcpartmci.t of 
Education 

Sacramento, California 
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INTRODUCTION 



What is responsible evaluation in the fields of adult, business, 
technical, and vocational education? How is educational evaluation con 
ducted? What types of evaluation skills do future educational leaders 
require? These are the questions which confronted the Division of Voca 
tional Education, University of California, Los Angeles, and the EPDA 
Leadership Program at UCLA. A variety of approaches were considered to 
meet the needs for training in evaluation on the part of those asking 
these questions. It was then determined that a workshop, coupled with 
some form of individual study, would most appropriately meet this need. 

Leadership development in evaluation for an age of accountability 
was selected as the theme for a \;orkshop and an individual s :udy ap- 
proach. This report describes, in bold strokes, an overview of the 
workshop and what it accomplished. Also included is a description of 
how the individual study approach is being implemented. 
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Intent 



The need for leadership development in evaluation was articulated in 
the early Fall of 1970 by several EPDA Fellows and staff members of the 
Division of Vocational Education. As a result of these requests, the 
Western Region AMIDS Evaluation Team examined a variety of approaches to 
leadership development which would make it possible for adult, business, 
technical and vocational educators to btcome knowledgeable in evaluation. By 
December of 1970, a proposed work plan encompassing a workshop and ?ndivi~ 
dual study approach was prepared. The following three principal intents 
were specified for the leadership development project: 

1. To provide participants with an introductory 
program in evaluative theory and methodology. 

2. To develop basic evaluative skills concerning 
techniques for evaluation of on-going pro- 
grams in adult, business, technical and 
vocational education. 

3. To provide participants with sufficient ref- 
erence materials and direction to enable 
them to undertake subsequent, individual 
study activities in the area of evaluation. 

Participant Needs 

While agreement on the value of knowledge about evaluation was 
acknowledged by personnel in the Division and by EPDA Fellows, the exact 
nature and extent of their needs was not known. It was known, however, 
that their backgrounds were diverse. Participants had heid, or now hold. 







2 



positions as administrators, teachers, curriculum developers, and training 
specialists. Because of this diversity, it was decided that an introductory 
v;orkshop, combining inductive and deductive experiences, followed by a 
suggested program of study would provide the best results. This approach 
ensured that all participants would be exposed to key concepts of evaluation. 

It also provided those individuals with a need, or a desire for, additional 
study with the necessary references. 

Program Plan 

A one-day workshop was planned for approximately twenty participants. 
During January, 1971, materials were developed, agendas were set and staff 
assignments made. Ic was decided that the workshop would be conducted in 
the Conference Room of the Division of Vocational Education in Santa Monica, 
California. This off-campus setting was far enough from UCLA to minimise 
interruptions. At the same time, tl;e location was not so far from the campus as 
to discourage attendance. The Conference Room comfortably seats twenty 
people and contains blackboards and a projection screen. Three smaller 
rooms in the same building were also available for group meetings. In 
retrospect, the location and facilities greatly enhanced the learning 
situation . 

Instructional Sequence and Methodology 

An indue tive -deduct ive learning approach was selected for both the 
workshop and the individual study phase of this leadership development 
activity. It was felt that such an approach, coupled with diverse instruc- 
tional methods, would result in the participant? developing a core of 
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knowledge concerning evaluation and would intensify a desire for further 
s tudy . 



Daring the first hour of the workshop, a 'Vase modtl" (Appendix B) 
was distributed. Participants were divided into throe groups of five to 
eight individuals. Each group had a discussion leader experienced in the 
field of evaluation. In the small group sessions, the part icipavits dealt 
inductively with the following questions: (1) I'^hat key functions in the 

case model should be evaluated? (2) Whrt are the advantages and disad- 
vantages of external versus internal evaluation methodology? (3) To what 
extent should the selection of aspects to be evaluated take into account 
the various groups which require feedback? Later in the morning, the par- 
ticipants and the w^orkshop staff met again in a geneial session to discuss 
the functions identified in the small groups. During this period, the 
workshop staff incorporated into the discussion a number of ’.T.portant as- 
pects of evaluation which had grown out of their AMIDS Evaluation Project. 

Dr. Stephen Klein, from the Center for Study of Evaluation, Univer- 
sity of California, Los Angeles, was the speaker for the workshop luncheon 
His presentation dealt with the roots of evaluation and its current status 
He also discussed an evaluative model which the Center had developed. A 
forty-minute question and answer period followed the presentation. 

The afternoon activities started with a team teaching session deal- 
ing with the concepts of "performance standards" and "standards of excel- 
lence." During this part of the workshop the staff again incorporated 
numerous key aspects of evaluation into the discussion. Particularly, 
emphasis was placed on the evaluative model which had been developed by 
the staff in the Western AMIDS Evaluation Project. 
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It became evident to the workshop staff, early in the afternoon, 
that the participants desired and were ready for additional discussion con* 
cerning "standards of excellence." The identification of such a need was 
possible because of the inductive -deductive approach being used. As a 
result, the small group sessions, which had been originally scheduled for 
the late afternoon, were cancelled to allow for additional co.isideration 
of “standards of excellence." The last hour of the v/orksliop was devoted 
to a presentation of evaluation techniques and methodologv appropriate to 
adult, business, technical and vocational education. 

Selected Aspects of Evaluation Presented 

During the program planning stage, key aspects of evaluation were 
selected for presentation during the workshop. Those key aspects were 
selected which would best meet the needs of the participants as identified 
in the workshop# The key aspects of evaluation presented at the workshop 
were as follows: 

1 . Purpose of evaluation 

2. Terminology of Evaluation 

3. Elements of Evaluation 
Key Indicators 

5. Constraints and other Considerations In Evaluation 

6. Standards of Excellence 

7. Types of Evaluation 

8. Chartering 

9. Internal and External Evaluation 



10. Absolute Comparisons 

11. Relativ.* Comparisons 

12. Differentiated Feedback 

13. Development of Evaluation Theory and Methodology 

14. Contributions of Leaders in the Field of Evaluation 



0 n»*fioing Individual Study 

Recognizing that a major objective or this leadership development 
activity was to provide participants with sufficient data and direction that 
they might undertake individual study of evaluation following the workshop, 
each attendee received an 82-page workbook developed by the staff. This 
^^valuation Workbook” was designed to present materials which would lielp 
the participants gain skills requisite to responsible program evaluation. 

It should be noted that, where necessary, authorization was obtained to 
reproduce materials on evaluation to be included in the workbook. The work- 
book contained the following materials: 



’’The Growth of Evaluation Methodology” 

by Gene V. Glass 

Western Region AMIDS Evaluation 

by James A. Farntcr, Jr., 
Ralph K. Sylvester, and 
Patrick J. Ueagraff 

'Western Region AMIDS Evaluation - A New’ Venture” 

by Patrick J. Weagraff 

"Indigenous Interactional Research” 

by James A. Farmer, Jr. 

Selected References on E/aluat'on 
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Participant Profile 



The following profile of the workshop participants can be constructed 
from two sources available to the workshop staff. The first is the infor- 
mation given on the cover sheet of the pre- or needs-assessment test (Appen- 
dix D) which was completed by the participants. The second source is a 
roster/data sheet of students attending UCLA through the EPDA Leadership 
Development Program in adult, business, technical, and vocational education. 

The characteristics of the attendees include the following; 

o A total of 21 such leaders and potential leaders 
participated in the workshop. Four of these 
participants, who are members of the Western 
AMIDS Evaluation Team, served as the workshop 
staff . 

o Twelve participants are EPDA Leadership Fellows 
presently attending UCLA and pursuing dr 
degrees in adult, business, technical, ano 
vocational education. Three participants are 
professors of Education at U CLA . One of the 
participants is Chief, Program Operations luit, 

Vocational Education Section, California State 
Department of Education, Two of the profei ^ors 
are further involved in duties within tiie 
Division of Vocational Education located on the 
UCLA campus. One of them is Division Director; 
the other, a project director for the Division. 



One remaining attendee is also a project director for 



the Division of Vocational Education. 



o All of the participants have at least one advance^ 
degree. Four presently hold doctorates. Of Che EPDA 
Fellows in attendance, ten are seeking the Ed.D., and 
two, the Ph.D., in education. 



o The twelve Fellows brought to the workshop a wide 
range of background experiences gained through posi- 
tions held prior to their acceptance of the leader- 
ship awards. Following is a summary list of those 
positions : 



State Supeivisor State Department of Education 

Assistant State 

Supervisor State Department of Education 

Specialist State Department of Education 



Coordinator, Curri- 
culum Deve lopment . . . S t ate Department of Education/ 

State University 

Director.... Post Secondary Vocational'/ 

Technical Institute 



Director, Vocational/ 

Academic 

Counseling.. State University 

Research Follow .State University 

o Participants indicated that their primary areas of 

educational interest and Involvement are the following: 

Adult Education 
Agricultural Education 
Business/office Education 
Distributive Education 
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Industrial Arts Education 
Trade and Industrial Education 
Vocational/academic Counseling 
Vocational/technical Education 

o Participants represent a total of seven states. Eleven 
are from California, with one each from Mississippi, 

Nei? Jersey, New York, Pennsylvania, Utah, and Wisconsin, 

o of the 17 participants, there were 16 males in attendance. 

Ident i fication and Analysis of Participant s ' Needs and Expectations 

The needs and expectations which the participants brought to the 
workshop can be generalized from analyses of two sources : (1) open-ended 

responses m?de by several participants on item 1 of the Participant Feed- 
back questionnaire (Appendix C) given to attendees for return by mail; and 
(2) errors mada by participants taking the pre- or needs -assessment test 
at the beginning of the workshop. 

Item 1 of the Partici^ ‘nt Feedback questionnaire was intended to 
elicit open-ended responses as to what each person expected and hoped to 
receive from the workshop. Questionnaires returned to the workshop staff 
indicated the following; 

o Participants expressed a variety of expectations and 
needs. Some persors identified reed tor instruction 
in methods and techniques for tne evaluation process. 

Others identified the need for conceptual learning 
concerning evaluation. The problem of accountability 
and evaluation in an ape of accountability were also 





identified , 



Incorrect responses on the forced -choice items and inadequate res- 
ponses on the open-ended items of the pre-test were taken as further 
indicators of need In specific aspects of evaluation considered important 
by the workshop staff. 

o Analysis of these items indicated the greatest need 
to be in connection with the following aspects of 
evaluation: selection of key indicators; evaluation 

of on-going programs; and identification of program 
intents and plansi 

Consequently, members of the workshop staff focused particular atten- 
tion on tViese aspects in both the inductive and the deductive segments of 
the workshop. 

Participants* Assessment of Wo rkshop 

The comments made by the participants in the small -group sessions 
and directly to the workshop staff during the day, expressed the participants' 
opinion that the workshop on evaluation was a valuable experience in 
leadership development. 

Additional feedback about the workshop v;as received on tho Partici- 
pant Feedback questionnaires (Appendix C), which were distributed to parti- 
cipants at the end of the workshop, and which were ationymously ccmpleted by 
the individual participants for return by mail. The following participant 
assessments were expressed on these questionnaires. 

o All of the participants responding indicated that 
the workshop evidenced good planning and/or 




organization. Several also stated .hat the materials 



used were appropriate and tha: the ii^structional 
methods used tended to maximize learner participation. 

o Many participants indicated that the workshop resulted 
in conceptual and/or methodological learnings. Several 
comments focused on the fact that they had gained 
knowledge of techniques useful in conducting evalua- 
tions. One indi'/id al stated that he received more 
than he had c ipccted in these regards. 

o There was a .vide range of responses to items 2 through 
7 of the questionnaire, all of which requested sug- 
gestions useful in improving future workshops. Some 
participants recommended that more workshops of a 
similar nature are needed. Others indicated possible 
ways to improve specific components of the workshop. 

One such recormendat ion proposed that an additional 
case study dealing with an actual evaluation in action 
be deve loped and utilized in future evaluation work- 
shops . 

o In generil, the participants were very positive in 

their reactions to the workshop. The follov;ing sample 
comments from the feedback questionnaires indicate such 
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satisfaction ; 



”For me^ just fine.*' 



*^It was excellent." 

*'Let's do it again,*' 

"I would like to thank everyone 
involved in the organization and 
presentation of the workshop." 

Some indication that favorable reactions were accompanied by learning 



gains was provided by a comparison of the results of the brief pre- and 
post’^tests (Appendix D) administered at the workshop. beaming gain seems 
to be particularly evident in relation to one of the aspects earlier iden- 
tified as an area of greatest need; namely, that of selection of key Indi- 
cators in evaluation. 

Implications for Future Workshops and Individual Study 



Based upon the experience of the workshop described in this report , 



the following implications for future workshops and for the individual study 
approach are evident: 



o Provision for some form of systematic follow-up to 

the workshop on evaluation, which was held February 9, 
1971, should be made. That workshop constituted 
a first step, pros^iding instruction in only those 
conceptual and methodological aspects of evaluation 
described earlier. Further help to the participants , 
particularly the EPDA Fellows presently involved in 
the leadership program at UCLA, in this vital area of 
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leadership development seems to be necessary. 

o The members of the Western AMIDS Evaluation Team, who 
acted as the workshop staff, are presently following 
up on some recommendations sugges':ed by the partici- 
pants. For example, the idea of developing and util- 
izing a naturalistic case model of an actual 
evaluation in action has been discussed and will be 
given further consideration. 

o The literature in the field of evaluation and 

accountability, together with the current limita- 
tions of funds in education, suggest that in-service 
leadership training of a variety of kinds is needed 
by personnel in adult, business, technical, and 
vocational education. 

o Both informal and formal feedback from the partici- 
pants in the workshop on evaluation indicate a great 
deal of interest in the general subject of, and 
variety of, aspects comprising evaluation. The number 
of requests from others for information about evalua- 
tion regarding adult, business, technical, and voca- 
tional education, which have been directed to the 
workshop staff, indicates that there is a growing 
pressure on the personnel in these fields to hecome 
operationally knowledgeable in evaluation for purposes 

13 
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of accountability. 



o It would seem that the inductive -deductive approach 
described in this report allows participants to 
share experiences and understandings which they bring 
to a workshop. Such personal experiences in and 
knowledge of evaluation can help to equip workshop 
participants for leadership in an age of accountability. 
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AGENDA 



Evaluation Workshop 



8:00 a«m. 


Welcoming Remarks 


M. L. Barlow 


8:30 a.m. 


Case Model 


P. J. Weagraff 


9:00 a.m. 


Small Group Sessions 






“Identifying Key Functions 
In Case Model to Evaluate” 


J. Farmer, Jr. 
P. Weagraff 
C. Chialtas 
J. Miller 


10:00 a.m. 


Coffee Break 




10:15 a.m. 


General Session 


J . Farmer , Jr . 


11:45 a.m. 


Break 




12:00 noon 


Luncheon 




2:00 p.m. 


General Session 


J . Farmer , Jr . 


2:45 p.m. 


Small Group Sessions 






“Performance Standards” 


J. Farmer, Jr. 
P, Weagraff 
C. Chialtas 
J. Miller 


3; 15 p.m 


General Session 






Group Reports 




4 ; 00 p.m. 


Panel 'Evaluative Techniques” 
and Audience Participation 


J. Farmer, Jr. 
P. Weagraff 
C. Chialtas 
J. Miller 
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Evaluation Workshop 



Case Model 

Occupational Skill Cei.ter Program Evaluation 

George Reynolds is Director of the Inner City Occupational Skill 
Center for Adults. The center which is three years old serves a population 
of 180,193. 

The Occupational Skill Center is one of 14 centers in the state, but 
it is the only one operated under contract by the local school district. 

The program at the Center primarily emphasizes the occupational preparation 
of adults with some attention given to adult basic education, vocational 
rehabilitation, and GED certification. Occupational preparation is offered 
in office practices, automotive, printing, allied health, and food services. 
The center also offers periodic training for power sewing machine operators, 
food cashiers, and electrical assemblers when local employers require such 
skills . 

Most of the funds used to support the Center's program come through 
the local school district. The district receives most of its funds f *om the 
State Department cf Education. Specific funding sources include the Voca- 
tional Education Act of 19b8 and the 1966 Adult Education Act. In addition, 
the district receives funds from the State Department of Human Resources, 
the Department of Labor and the Veterans Administration. l^ocal taxes and 
tuition assessment of some participants also are sources of revenue. 

The total Center progr.im has both day and e\^ening componeiits with a 
combined enrollment of approximately 303 students, both full and part-time. 
Most of the participants (737.) are enrolled in the day program, Half of 

o 
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ADC mothers. The 



the students are publi- aid recipients, many of whom are 
age range is 18-64 years. There are more women than men participants. Most 
of the participants, black and white, have low socio-economic backgrounds. 

A few participants of foreign birth are enrolled in the program for the 

purpose of learning to read and VTrite English. 

The director of th. Center is on a full time appointment and super- 
vises both the day and evening programs. The day teaching staff consists 
of 26 full-time and 10 part-time instructors. The evening staff includes 
four of the day staff members and five additional ones. Only one of the 
staff has not had prior experience in the public schools. Mone of ^he staff 
had experience in any manpower training activities prior to joining the 

Centers . 

in addition to the instructors, the staff includes a full-time public 
aid case worker and two counselors. One counselor is concerned with basic 
education; the other is a vocational coordinator who provides help in job 
placement and works with students in establishing realistic work goals. 

The instructional program is diverse, being designed in many cases 
to meet individual student needs. Classes operate from 7 a.m. to 9 p.m. 

For GED aspirants, there is a full-time program which runs from 9:20 a.m. 
to 2-.20 p.m. five days a week. Occupational preparation occurs between 

7 a.m. and 4 p.m. with students being released io take the basic education 

/-< ' Tr^ rronofiil tl'lPl'G tJtC HO St d 3 1* d 

they require for occupational proficvency. Tn general, 

beginning and ending dates for any program. 

A most interesting aspect of the program is the close relationship 

the Center maintains with local employers. Mr. Reynolds has over 20 



active advisory groups. 



The Center has several activities it regards as unique. To enhance 



the likelihood for potential participants (especially ADC mothers) with 
pre-school children to enroll in the program a day care program has been 
established. This program also serves to train para-professionals in early 
childhood services. 

Due to the complexities involved in the administration and financing 
of the Center, the director feels accountable, in different ways, to several 
audiences. For example, the local district is primarily interested in the 
smooth operation of the Center. The Department of Human Resources is most 
interested in the rate of student placement on jobs. The teaching staff 
of the Center is interested in program revision and improvement. The local 
employers look to the Center for a pool of trained manpower. The State 
Department of Education views evidence of fiscal accountability as important 
data on which they evaluate the Center. 

Given the possibility that evaluative data about the Center could 
be obtained and that Mr. Reynolds turned to your evaluative teams for 
assistance, what appropriate information would you provide him with for 
decision-making as an administrator? As a first step in chi process, he 
has asked your team to provide answers to the following questions: 



1. What aspects of the Center should be evaluated? (List) 

2. Who should do what in the evaluation of those aspects 
which you specified in #1? 

3. To what extent should the selection of aspects to be 
evaluated t ake into account the various audiences 

which require feedback? (See page 3, 2nd paragraph) Why? 



o 
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EVALUATIVE RESEARCH WORKSHOP 



Conducted 

for 

Division of Vocational Education 
University of California, Los Angeles 

Dr, James A. Farmer, Project Director, Western AMIDS Evaluation 
Patrick J. Weagraff, Christ T. Chialtas, and Joseph A. Miller, 
Research Specialists and EPDA Leadership Fellows 



Participant Feedback 



Please answer lionestly and candidly the following questions about the 
workshop on evaluative research you attended February 9, 1971, Please 
give adequate thought to each question before making your response. Your 
responses will be used constructively by the evaluative research team in 
its efforts to modify such workshops for the benefit of future participants 
by assessing strengths and weaknesses, A self-addressed, stamped envelope 
is attached for your convenience in returning your thoughts to us. 



What had you expected and hoped to receive from the workshop? 



2. What did you actually experience during the workshop (please be as 
specific as possible)? 



3. What inputs or effort did you see going into the program? 




A. What outcomes or results did you see coming out of the program (personally 
and for the group)? 



5. Given the extent of the need for such a workshop, how adequate were 
the outcomes? 



6. ^That recommendations can you make for improving the workshop? 



7. Additional comments. 
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EVALUATIVE RESEARCH WORKSHOP 



Conducted 

for 

Division of Vocational Education 
University of California, Los Angeles 

by 

Dr, James A. Farmer, Jr., Project Director, Western AMIDS Evaluation 
Patrick J. Weagraff, Christ T, Chlaltas, and Joseph A. Miller, 
Rfisearch Specialists and EPDA Leadership Fellows 



Pre- and Post-Tests 



This evaluation instrument is aimed at indicating your understanding of 
program evaluation, both generally and as embodied in the program evalu- 
ation model of the Western AMDS evaluation team. It will be administered 
in two parts - the pre-test, attached to this cover sheet and to be completed 
at this time, and the post-test, to be given to you at the end of this 
workshop. 

Please provide the following lnfo*mation about yourself. Onl y the Evalu - 
c clon team will see your paper . The responses will be analyzed and used 
in differentiated reports to interested, appropriate parties. NOTE: As we 

are not asking for your name, please make sure to indicate your date of birth, 
here and on the post-test, so that your pre- and post-test can be paired. 



Date of Workshop Date of dirth 

Sex State of Residence 

Position (last held. If student) 

Degrees: Obtained , Sought 

Vocational Field (Ex: Agricultural Education) 
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PRE-TEST 



Dir:>ctions: W^rite in the space to the left of ea'^l. item if the state- 

ment is true based on current thinking in evaluative research; 
write ’V’’ if the statement is false. 



1. Evaluation should seek to determine the degrees of success in 
achieving only the major objectives of a program* 

2. The ’’best laid plans of mice and men" often do not see total 
fruition; an evaluator must therefore concern himself only with 
what actually enters, is processed, and results from the system 
being evaluated. 

3. Since the evaluation of educational programs consists largely of 
course ex£:.ns , learner self -apprais al and satisfaction forms, and 
analyses of enrollment, any effective evaluative research model 
would best structure itself around these feedback devices. 

4. Feedback from the evaluator to key personnel of an on-going pro- 
gram is illegitimate in that it biases what the evaluator, in 
fact a researcher, is attempting to measure. 

5. Evaluation, like a photograph, can merely seek to describe past 
events or conditions; in particular, it can at best show past 
successes/failures and achievements of an on-going program- 

6. Objectives can be immediate, intermediate, or long-run with 
respect to duration ? importance. 

7. Since evaluation des es the present operation of a program, 
background research with respect to goals, intents, and assump- 
tions are not too important. 

8. The determination of whether or not an objective could be better 
or less costly achieved through an alternative procedure is a 
value judgment inconsistent with the proper role of an evaluator. 

9. "The fact that our office successfully referred over 200 appli- 
cants to appropriate manpower agencies last month for training 
Is a good Indication of the adequacy of our guidance service." 

10. The comparison between the program being evaluated and like or 
related programs provides the sum total of wholistic comparative 
data for the evaluator. 

11. Evaluation can be likened to the constructing of a balance sheet 
in that the results constitute a static picture of conditions at 
a given point. 
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12. In evaluation, by its nature a subjective process, the evaluator 
must seek support for his measurement criteria so as to help 
validate them. 



Directions; Respond to the following open-ended items within the space 
provided even though you may need more. 



13. Identify or list those factors which you consider to ba critical for 

the evaluation of an on-going program in adult, vocational or business 
education. 



14. In one sentence identify what you perceive to be the major purpose of 
an evaluation of an cn-going program. 



15. List the names of some individuals generally recognized as being leaders 
in the field of evaluative research. 
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POST-TEST 



Date of Birth 



Directions; Write in the space to the left of each item if the state- 
ment is true based on current thinking in evaluative research; 
write "F" if the statement is false. 




1. Since it is impossible to collect data on every aim or facet 
of an on-going program, evaluative efforts must be selective 
in their identification of specifics worth evaluating. 

2 . An intended, but un-realized, input to a program is inadmiss- 
able ’’evidence" in the judging of the success/failure of that 
program. 

3. Given that we're entering an age of accountability in education, 
there is still a lack of methodological convention in evaluative 
research of on-going programs. 

4. In the course of evaluating a program, much is learned about 
why and where it has failed; such evidence for making decisions 
can Justifiably be given to administration through feedback 
from the evaluator. 



5. Any evaluative effort worth conducting should have as an objective 
the identification of emergent or indicated future needs uf the 
program being evaluated. 

6. Objectives can be hierarchical in nature in that one may be 
all-inclusive of those supportive and subordinate to ic. 

7. Before one can make unblindered observations of aspects of a 
program, awareness of the assumptions, plans, and goals, etc., 
recognized by administration as relevant, is necessary. 

8. The evaluator, in order to maintain his researcher posture, 
should stay away from such administrative measures as cost- 
benefit analysis. 

9. While a measure of results Is important, the total assessed need 
for such performance provides a baseline from which to judge the 
adequacy of the results. 



10 . 



A kind of evaluative comparison can be made between the program 
evaluated and a hypothetical, excellent one. 



ERIC CIcaripQhouse 

JUL 7 m 
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on Adult Eilucrtion 



